{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 1.网络结构"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "VGGNet由多个VGG块组成，每个VGG块中，包含若干个卷积层和一个池化层"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![image-20240809092537792](https://zyc-learning-1309954661.cos.ap-nanjing.myqcloud.com/machine-learning-pic/image-20240809092537792.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "VGGNet有多个系列，如：VGG-11, VGG-16, VGG-19。后面的数字意味着模型中包含权重参数的层个数（卷积层+全连接层）"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 2.VGG块"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "VGG块包含多个卷积层+一个池化层。\n",
    "\n",
    "1. 如何确定VGG块中卷积层的个数？对于每个VGGNet的系列，其块中卷积层个数是定义好的\n",
    "   - VGG-16：\n",
    "     - 第一和第二个 VGG 块：每个块包含 2 个卷积层。\n",
    "     - 第三和第四个 VGG 块：每个块包含 3 个卷积层。\n",
    "     - 第五个 VGG 块：包含 3 个卷积层。\n",
    "   - VGG-19:\n",
    "     - 第一和第二个 VGG 块：每个块包含 2 个卷积层。\n",
    "     - 第三、第四和第五个 VGG 块：每个块包含 4 个卷积层。\n",
    "2. 每个块中的卷积层属性是否一致？一致，都为3*3，padding=1，且每个卷积层的输入和输出通道数分别相同"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Sequential(\n",
      "  (0): Conv2d(1, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))\n",
      "  (1): ReLU()\n",
      "  (2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))\n",
      "  (3): ReLU()\n",
      "  (4): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))\n",
      "  (5): ReLU()\n",
      "  (6): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)\n",
      ")\n"
     ]
    }
   ],
   "source": [
    "import torch\n",
    "from torch import nn\n",
    "\n",
    "'''\n",
    "功能：\n",
    "    构造块\n",
    "参数:\n",
    "    conv_num:       卷积层个数\n",
    "    in_channels:    卷积层输入通道数\n",
    "    out_channels:   卷积层输出通道数\n",
    "'''\n",
    "def block(conv_num, in_channels, out_channels):\n",
    "    layer = []\n",
    "    for _ in range(conv_num):\n",
    "        layer.append(nn.Conv2d(in_channels, out_channels, kernel_size=3, padding=1))\n",
    "        layer.append(nn.ReLU())\n",
    "        in_channels = out_channels\n",
    "    layer.append(nn.MaxPool2d(kernel_size=2, stride=2))\n",
    "    return nn.Sequential(*layer)\n",
    "\n",
    "print(block(3, 1, 64))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 3.VGG-16的实现"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "VGG-16网络结构为：\n",
    "\n",
    "- 第一层和第二层:\n",
    "  - 卷积层1：64 个通道\n",
    "  - 卷积层2：64 个通道\n",
    "  - 池化层：后接 2x2 最大池化\n",
    "- 第三层和第四层:\n",
    "  - 卷积层3：128 个通道\n",
    "  - 卷积层4：128 个通道\n",
    "  - 池化层：后接 2x2 最大池化\n",
    "- 第五层到第七层:\n",
    "  - 卷积层5：256 个通道\n",
    "  - 卷积层6：256 个通道\n",
    "  - 卷积层7：256 个通道\n",
    "  - 池化层：后接 2x2 最大池化\n",
    "- 第八层到第十层:\n",
    "  - 卷积层8：512 个通道\n",
    "  - 卷积层9：512 个通道\n",
    "  - 卷积层10：512 个通道\n",
    "  - 池化层：后接 2x2 最大池化\n",
    "- 第十一层到第十三层:\n",
    "  - 卷积层11：512 个通道\n",
    "  - 卷积层12：512 个通道\n",
    "  - 卷积层13：512 个通道\n",
    "  - 池化层：后接 2x2 最大池化"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "----------------------------------------------------------------\n",
      "        Layer (type)               Output Shape         Param #\n",
      "================================================================\n",
      "            Conv2d-1         [-1, 64, 224, 224]             640\n",
      "              ReLU-2         [-1, 64, 224, 224]               0\n",
      "            Conv2d-3         [-1, 64, 224, 224]          36,928\n",
      "              ReLU-4         [-1, 64, 224, 224]               0\n",
      "         MaxPool2d-5         [-1, 64, 112, 112]               0\n",
      "            Conv2d-6        [-1, 128, 112, 112]          73,856\n",
      "              ReLU-7        [-1, 128, 112, 112]               0\n",
      "            Conv2d-8        [-1, 128, 112, 112]         147,584\n",
      "              ReLU-9        [-1, 128, 112, 112]               0\n",
      "        MaxPool2d-10          [-1, 128, 56, 56]               0\n",
      "           Conv2d-11          [-1, 256, 56, 56]         295,168\n",
      "             ReLU-12          [-1, 256, 56, 56]               0\n",
      "           Conv2d-13          [-1, 256, 56, 56]         590,080\n",
      "             ReLU-14          [-1, 256, 56, 56]               0\n",
      "           Conv2d-15          [-1, 256, 56, 56]         590,080\n",
      "             ReLU-16          [-1, 256, 56, 56]               0\n",
      "        MaxPool2d-17          [-1, 256, 28, 28]               0\n",
      "           Conv2d-18          [-1, 512, 28, 28]       1,180,160\n",
      "             ReLU-19          [-1, 512, 28, 28]               0\n",
      "           Conv2d-20          [-1, 512, 28, 28]       2,359,808\n",
      "             ReLU-21          [-1, 512, 28, 28]               0\n",
      "           Conv2d-22          [-1, 512, 28, 28]       2,359,808\n",
      "             ReLU-23          [-1, 512, 28, 28]               0\n",
      "        MaxPool2d-24          [-1, 512, 14, 14]               0\n",
      "           Conv2d-25          [-1, 512, 14, 14]       2,359,808\n",
      "             ReLU-26          [-1, 512, 14, 14]               0\n",
      "           Conv2d-27          [-1, 512, 14, 14]       2,359,808\n",
      "             ReLU-28          [-1, 512, 14, 14]               0\n",
      "           Conv2d-29          [-1, 512, 14, 14]       2,359,808\n",
      "             ReLU-30          [-1, 512, 14, 14]               0\n",
      "        MaxPool2d-31            [-1, 512, 7, 7]               0\n",
      "          Flatten-32                [-1, 25088]               0\n",
      "           Linear-33                 [-1, 4096]     102,764,544\n",
      "             ReLU-34                 [-1, 4096]               0\n",
      "          Dropout-35                 [-1, 4096]               0\n",
      "           Linear-36                 [-1, 4096]      16,781,312\n",
      "             ReLU-37                 [-1, 4096]               0\n",
      "          Dropout-38                 [-1, 4096]               0\n",
      "           Linear-39                   [-1, 10]          40,970\n",
      "          Softmax-40                   [-1, 10]               0\n",
      "================================================================\n",
      "Total params: 134,300,362\n",
      "Trainable params: 134,300,362\n",
      "Non-trainable params: 0\n",
      "----------------------------------------------------------------\n",
      "Input size (MB): 0.19\n",
      "Forward/backward pass size (MB): 218.77\n",
      "Params size (MB): 512.32\n",
      "Estimated Total Size (MB): 731.28\n",
      "----------------------------------------------------------------\n"
     ]
    }
   ],
   "source": [
    "import torch\n",
    "from torch import nn\n",
    "import torchsummary\n",
    "\n",
    "class VGGNet_16(nn.Module):\n",
    "    def __init__(self):\n",
    "        super(VGGNet_16, self).__init__()\n",
    "        self.feature = nn.Sequential(\n",
    "            block(2, 1, 64),\n",
    "            block(2, 64, 128),\n",
    "            block(3, 128, 256),\n",
    "            block(3, 256, 512),\n",
    "            block(3, 512, 512),\n",
    "            nn.Flatten()\n",
    "        )\n",
    "        self.fc = nn.Sequential(\n",
    "            nn.Linear(in_features=512*7*7, out_features=4096),\n",
    "            nn.ReLU(),\n",
    "            nn.Dropout(),\n",
    "            nn.Linear(in_features=4096, out_features=4096),\n",
    "            nn.ReLU(),\n",
    "            nn.Dropout(),\n",
    "            nn.Linear(in_features=4096, out_features=10),\n",
    "            nn.Softmax(dim=1)\n",
    "        )\n",
    "\n",
    "    def forward(self, x):\n",
    "        x = self.feature(x)\n",
    "        x = self.fc(x)\n",
    "        return x\n",
    "    \n",
    "device = 'cuda' if torch.cuda.is_available() else 'cpu'\n",
    "net = VGGNet_16().to(device)\n",
    "torchsummary.summary(net, input_size=(1,224,224)) # 查看网络结构"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 4.VGG-11的实现"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- Block1：3*3卷积×1+最大池化+relu（输入通道：1，输出通道：64）\n",
    "- Block2：3*3卷积×1+最大池化+relu（输入通道：64，输出通道：128）\n",
    "- Block3：3*3卷积×2+最大池化+relu（输入通道：128，输出通道：256）\n",
    "- Block4：3*3卷积×2+最大池化+relu（输入通道：256，输出通道：512）\n",
    "- Block5：3*3卷积×2+最大池化+relu（输入通道：512，输出通道：512）\n",
    "- classifier：fc×3+softmax"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "----------------------------------------------------------------\n",
      "        Layer (type)               Output Shape         Param #\n",
      "================================================================\n",
      "            Conv2d-1         [-1, 16, 224, 224]             160\n",
      "              ReLU-2         [-1, 16, 224, 224]               0\n",
      "         MaxPool2d-3         [-1, 16, 112, 112]               0\n",
      "            Conv2d-4         [-1, 32, 112, 112]           4,640\n",
      "              ReLU-5         [-1, 32, 112, 112]               0\n",
      "         MaxPool2d-6           [-1, 32, 56, 56]               0\n",
      "            Conv2d-7           [-1, 64, 56, 56]          18,496\n",
      "              ReLU-8           [-1, 64, 56, 56]               0\n",
      "            Conv2d-9           [-1, 64, 56, 56]          36,928\n",
      "             ReLU-10           [-1, 64, 56, 56]               0\n",
      "        MaxPool2d-11           [-1, 64, 28, 28]               0\n",
      "           Conv2d-12          [-1, 128, 28, 28]          73,856\n",
      "             ReLU-13          [-1, 128, 28, 28]               0\n",
      "           Conv2d-14          [-1, 128, 28, 28]         147,584\n",
      "             ReLU-15          [-1, 128, 28, 28]               0\n",
      "        MaxPool2d-16          [-1, 128, 14, 14]               0\n",
      "           Conv2d-17          [-1, 128, 14, 14]         147,584\n",
      "             ReLU-18          [-1, 128, 14, 14]               0\n",
      "           Conv2d-19          [-1, 128, 14, 14]         147,584\n",
      "             ReLU-20          [-1, 128, 14, 14]               0\n",
      "        MaxPool2d-21            [-1, 128, 7, 7]               0\n",
      "          Flatten-22                 [-1, 6272]               0\n",
      "           Linear-23                 [-1, 4096]      25,694,208\n",
      "             ReLU-24                 [-1, 4096]               0\n",
      "          Dropout-25                 [-1, 4096]               0\n",
      "           Linear-26                 [-1, 4096]      16,781,312\n",
      "             ReLU-27                 [-1, 4096]               0\n",
      "          Dropout-28                 [-1, 4096]               0\n",
      "           Linear-29                   [-1, 10]          40,970\n",
      "          Softmax-30                   [-1, 10]               0\n",
      "================================================================\n",
      "Total params: 43,093,322\n",
      "Trainable params: 43,093,322\n",
      "Non-trainable params: 0\n",
      "----------------------------------------------------------------\n",
      "Input size (MB): 0.19\n",
      "Forward/backward pass size (MB): 31.48\n",
      "Params size (MB): 164.39\n",
      "Estimated Total Size (MB): 196.06\n",
      "----------------------------------------------------------------\n"
     ]
    }
   ],
   "source": [
    "import torch\n",
    "from torch import nn\n",
    "import torchsummary\n",
    "\n",
    "ratio = 4\n",
    "\n",
    "class VGGNet_11(nn.Module):\n",
    "    def __init__(self):\n",
    "        super(VGGNet_11, self).__init__()\n",
    "        self.feature = nn.Sequential(\n",
    "            block(1, 1, 64//ratio),\n",
    "            block(1, 64//ratio, 128//ratio),\n",
    "            block(2, 128//ratio, 256//ratio),\n",
    "            block(2, 256//ratio, 512//ratio),\n",
    "            block(2, 512//ratio, 512//ratio),\n",
    "            nn.Flatten()\n",
    "        )\n",
    "        self.fc = nn.Sequential(\n",
    "            nn.Linear(in_features=512//ratio*7*7, out_features=4096),\n",
    "            nn.ReLU(),\n",
    "            nn.Dropout(),\n",
    "            nn.Linear(in_features=4096, out_features=4096),\n",
    "            nn.ReLU(),\n",
    "            nn.Dropout(),\n",
    "            nn.Linear(in_features=4096, out_features=10),\n",
    "            nn.Softmax(dim=1)\n",
    "        )\n",
    "\n",
    "    def forward(self, x):\n",
    "        x = self.feature(x)\n",
    "        x = self.fc(x)\n",
    "        return x\n",
    "    \n",
    "device = 'cuda' if torch.cuda.is_available() else 'cpu'\n",
    "net = VGGNet_11().to(device)\n",
    "torchsummary.summary(net, input_size=(1,224,224)) # 查看网络结构"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 5.训练"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "硬件性能不够，暂时跑不了原模型，把VGG-11的卷积层通道数缩小至1/4"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "epoch:1, loss:0.0162729197204113, accuracy:0.5609\n",
      "epoch:2, loss:0.014273237462838491, accuracy:0.6459\n",
      "epoch:3, loss:0.013814703238010406, accuracy:0.7584\n",
      "epoch:4, loss:0.012930014447371165, accuracy:0.8145\n",
      "epoch:5, loss:0.012747807212670645, accuracy:0.8383\n",
      "epoch:6, loss:0.012638463634252549, accuracy:0.8431\n",
      "epoch:7, loss:0.012579616608222325, accuracy:0.8473\n",
      "epoch:8, loss:0.01251416520078977, accuracy:0.8615\n",
      "epoch:9, loss:0.01244802205959956, accuracy:0.8607\n",
      "epoch:10, loss:0.01242356671889623, accuracy:0.8511\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAiMAAAGdCAYAAADAAnMpAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8pXeV/AAAACXBIWXMAAA9hAAAPYQGoP6dpAAAvzUlEQVR4nO3de1zUdb7H8fdwBwW8oIiEWWblXQPBa3ahTJO03DKzdD3VPuqoaZzdI1bq6XSSytWlTTfTs7XnnDIte6x5y2qpLFPSNLaLecu7BoIag6DMMPM7f7jMioIyiHwZ5vV8PH6PZr6/y3yGIX9vvr/v7zs2y7IsAQAAGBJgugAAAODfCCMAAMAowggAADCKMAIAAIwijAAAAKMIIwAAwCjCCAAAMIowAgAAjAoyXUBNuN1uHTlyRJGRkbLZbKbLAQAANWBZloqLi9W2bVsFBFTf/+ETYeTIkSNKSEgwXQYAAKiFgwcP6oorrqh2vU+EkcjISEln3kxUVJThagAAQE3Y7XYlJCR4zuPV8YkwUnFpJioqijACAICPudgQCwawAgAAowgjAADAKMIIAAAwyifGjNSEy+WS0+k0XQa8FBgYqKCgIG7ZBgA/1ijCyMmTJ3Xo0CFZlmW6FNRCRESE4uLiFBISYroUAIABPh9GXC6XDh06pIiICLVq1Yq/sH2IZVlyOBwqKCjQ3r171bFjxwtOigMAaJx8Pow4nU5ZlqVWrVopPDzcdDnwUnh4uIKDg7V//345HA6FhYWZLgkAUM8azZ+h9Ij4LnpDAMC/cRYAAABGEUYAAIBRhBFDbrrpJk2ZMsV0GQAAGEcYAQAARvn83TQAgIbLsiydOnVKdrtddrtdRUVFstvtcjqdnhsPbDZbpaWqNm+2vZQ2b/cPDg5WeHi4IiIiFBYWxs0UtdT4wohlSaWlZl47IkKqxS/iiRMnNHnyZK1cuVJlZWUaNGiQ/vjHP6pjx46SpP3792vixIlav369HA6H2rdvr9mzZ2vo0KE6ceKEJk6cqI8++kgnT57UFVdcoaeeekrjx4+v63cHAJVYlqWysjJPwDg7bJz9vLy83HSp9cJms3mCybn/re5xeHg4dxSqMYaR0lKpaVMzr33ypNSkide7/frXv9auXbu0YsUKRUVFaerUqRo6dKi2bdum4OBgTZgwQQ6HQ59//rmaNGmibdu2qek/3uP06dO1bds2ffDBB4qJidHu3bt16tSpun5nAPxQWVlZpYBxbtCw2+1yOBw1OlZERISioqIUHR2tqKgohYaGyrIsz8zZFY8v1HZ2e232qau2s9udTqdKS0vlcDhkWZZKS0tV6uUfxGFhYRcMKxVtZz8PDAz06jUausYXRnxMRQj58ssv1a9fP0nSW2+9pYSEBC1fvlz33nuvDhw4oJEjR6pbt26SpKuvvtqz/4EDB9SrVy8lJSVJktq3b1/v7wGA73E4HOcFjKKiIhUXF3vaysrKanSs8PDwSkGjYjn7eVBQ4z7dlJeX69SpUyotLfX89+zHVbWdPn1aknT69GnP45oKCQm5YHCp6nFwcPDleOt1ovH9dkREnOmhMPXaXvrxxx8VFBSklJQUT1vLli113XXX6ccff5QkPfHEE3r88cf10UcfKTU1VSNHjlT37t0lSY8//rhGjhyprVu36vbbb9eIESM8oQaAf3I6nRe8bGK322t88gsLC6s2YFQ8bsgnufoSFBSkyMhIRUZG1ngft9tdZVi5UIg5deqU56s0HA6HfvnlF69qvFCA6dSpk6Kiomrx7i9d4wsjNlutLpU0ZI888ogGDx6s1atX66OPPlJmZqbmzJmjSZMmaciQIdq/f7/WrFmjjz/+WLfeeqsmTJig3//+96bLBnAZlJeXVxkuzn5e00u1oaGhlYLG2QGj4r98geXlExAQoCZNmqiJF+csy7J0+vTp80LLxXph3G53pd+dqrRt25Yw4q86deqk8vJyffXVV54ejWPHjmnHjh3q3LmzZ7uEhAQ99thjeuyxxzRt2jQtWrRIkyZNkiS1atVK48aN07hx4zRw4ED97ne/I4wA9cCyLJWXl8vpdKq8vLzScm7bxZ7XZDun01njSyfBwcEXvGwSHR2t0NDQy/wTQl2rGCQbHh6uli1b1mifip6UqgLM2T0upoKIRBgxrmPHjho+fLgeffRRvfbaa4qMjFRGRobi4+M1fPhwSdKUKVM0ZMgQXXvttTpx4oQ+/fRTderUSZI0Y8YMJSYmqkuXLiorK9OqVas86wB/UvGXn9PpPG+p6qR+qcGgvLxcLpfLyHsNCgq66BgNbjNFBZvNptDQUIWGhqp58+amy6kSYaQBeOONNzR58mQNGzZMDodDN954o9asWeO5DutyuTRhwgQdOnRIUVFRuuOOO/SHP/xB0plBTNOmTdO+ffsUHh6ugQMHasmSJSbfDuBhWZbcbvcFA0JN1l0sZDidTmPBoEJAQICCgoIUFBSk4OBgz+Oqnl9o3cWeV1znJ2igMbFZFfcoNWB2u13R0dEqKio6rxvp9OnT2rt3r6666iq+ft5H8Rk2PC6XS4WFhcrPz9fp06drHRCcTqdM/BNTcRKvWM49qVcXDrwNBmc/Z64I4HwXOn+fjZ4RwM+dPn1a+fn5ysvL8ywFBQV13tNgs9kqBYSzQ0Jt2i60Lb0GgG8hjAB+wrIs2e12T+DIz8/Xzz//XO2tgWFhYYqNjVWTJk2qPPFXFxCqWxcQEEBIAFAlwgjQCFVcZjk7eOTl5VV7y2d0dLTatGlTaYmOjiY8AKgXhBHAx5WVlZ13meXo0aNVXmYJCAhQq1at1KZNG8XGxiouLk6xsbEKDw83UDkAnEEYAXyEZVkqLi6uFDry8vJ04sSJKrcPCQk5r7ejVatWjX5abgC+h3+VgAbI7XZXeZmlui/gioqKOi94NGvWjMssAHwCYQQwzOFwVHmZpaqvXbfZbFVeZomoxfciAUBDQRgB6ollWTp58uR5l1mOHz9e5fYhISGKjY2t1NvRunVrLrMAaHT4Vw24DNxut44dO3beZZaSkpIqt4+MjDzvMkvz5s25zALALxBGgDpUWFionJwcfffdd3I4HOett9lsiomJ8VxmqQge3nxrJwA0NoQReDidTs/34aDmLMvS3r17lZOTo127dnnag4ODq7zMws8YACrjyxQMWrt2rQYMGKBmzZqpZcuWGjZsmH766SfP+kOHDmn06NFq0aKFmjRpoqSkJH311Vee9StXrlTv3r0VFhammJgY3X333Z51NptNy5cvr/R6zZo101/+8hdJ0r59+2Sz2bR06VINGjRIYWFheuutt3Ts2DGNHj1a8fHxioiIULdu3fT2229XOo7b7dZLL72ka665RqGhoWrXrp2ef/55SdItt9yiiRMnVtq+oKBAISEhys7OrosfW4NRXl6u3Nxcvfbaa/q///s/TxC57rrrNG7cOGVkZOjhhx/WnXfeqcTERMXHxxNEAKAKteoZmT9/vmbPnq28vDz16NFDr7zyipKTk6vdPisrS6+++qoOHDigmJgY/epXv1JmZuZl+VI0y7LkdDrr/Lg1ERwc7NU1/pKSEqWnp6t79+46efKkZsyYobvvvlu5ubkqLS3VoEGDFB8frxUrVqhNmzbaunWr3G63JGn16tW6++679fTTT+t///d/5XA4tGbNGq9rzsjI0Jw5c9SrVy+FhYXp9OnTSkxM1NSpUxUVFaXVq1froYceUocOHTyf8bRp07Ro0SL94Q9/0IABA/Tzzz9r+/btkqRHHnlEEydO1Jw5cxQaGipJevPNNxUfH69bbrnF6/oaopKSEn399dfavHmzZwxIcHCwevbsqZSUFLVs2dJwhQDgW7wOI0uXLlV6eroWLFiglJQUZWVlafDgwdqxY4dat2593vaLFy9WRkaGXn/9dfXr1087d+7Ur3/9a9lsNs2dO7dO3sTZnE6nMjMz6/y4NTFt2jSFhITUePuRI0dWev7666+rVatW2rZtmzZs2KCCggJt3rxZLVq0kCRdc801nm2ff/553X///Xr22Wc9bT169PC65ilTpuiee+6p1Pbb3/7W83jSpEn68MMP9c477yg5OVnFxcV6+eWXNW/ePI0bN06S1KFDBw0YMECSdM8992jixIl6//33dd9990mS/vKXv3g+c19WUFCgnJwcffvtt57bbqOiopScnKwbbriBWUwBoJa8DiNz587Vo48+qvHjx0uSFixYoNWrV+v1119XRkbGedtv2LBB/fv31wMPPCBJat++vUaPHl3pcoO/2rVrl2bMmKGvvvpKhYWFnl6PAwcOKDc3V7169fIEkXPl5ubq0UcfveQakpKSKj13uVyaNWuW3nnnHR0+fFgOh0NlZWWeeSx+/PFHlZWV6dZbb63yeGFhYXrooYf0+uuv67777tPWrVv1/fffa8WKFZdcqwmWZWnPnj3auHFjpUtobdu2VZ8+fdS5c2cFBgYarBAAfJ9XYcThcGjLli2aNm2apy0gIECpqanauHFjlfv069dPb775pjZt2qTk5GTt2bNHa9as0UMPPVTt65SVlamsrMzz3G6317jG4ODgSvXVJ2/HA6SlpenKK6/UokWL1LZtW7ndbnXt2lUOh+Oif2VfbL3NZpNlWZXaqrp8de5dHLNnz9bLL7+srKwsdevWTU2aNNGUKVM8d4bU5K//Rx55RD179tShQ4f0xhtv6JZbbtGVV1550f0akvLycn377bfKyclRQUGBp71Tp07q06ePEhISfL6nBwAaCq/CSGFhoVwul2JjYyu1x8bGesYMnOuBBx5QYWGhBgwYIMuyVF5erscee0xPPfVUta+TmZlZ6fKDN2w2m1eXSkw5duyYduzYoUWLFmngwIGSpPXr13vWd+/eXf/93/+t48ePV9k70r17d2VnZ3t6qM7VqlUr/fzzz57nu3btqnYq8bN9+eWXGj58uB588EFJZwar7ty5U507d5YkdezYUeHh4crOztYjjzxS5TG6deumpKQkLVq0SIsXL9a8efMu+roNxcmTJz3jQSp+XiEhIerVq5dSUlLUvHlzwxUCQONz2W/t/eyzzzRr1iz96U9/UkpKinbv3q3Jkyfrueee0/Tp06vcZ9q0aUpPT/c8t9vtSkhIuNyl1qvmzZurZcuWWrhwoeLi4nTgwIFKl7lGjx6tWbNmacSIEcrMzFRcXJy++eYbtW3bVn379tXMmTN16623qkOHDrr//vtVXl6uNWvWaOrUqZLO3NUyb9489e3bVy6XS1OnTq1Rz03Hjh21bNkybdiwQc2bN9fcuXOVn5/vCSNhYWGaOnWq/v3f/10hISHq37+/CgoK9MMPP+jhhx/2HKdiIGuTJk0q3eXTUOXn53vmB6n4ttvo6GjPeJDLMdgaAHCGV2EkJiZGgYGBys/Pr9Sen5+vNm3aVLnP9OnT9dBDD3n+iu7WrZtKSkr0m9/8Rk8//bQCAs6/uzg0NNRzJ0ZjFRAQoCVLluiJJ55Q165ddd111+mPf/yjbrrpJkln/hr/6KOP9G//9m8aOnSoysvL1blzZ82fP1+SdNNNN+ndd9/Vc889pxdeeEFRUVG68cYbPcefM2eOxo8fr4EDB6pt27Z6+eWXtWXLlovW9cwzz2jPnj0aPHiwIiIi9Jvf/EYjRoxQUVGRZ5vp06crKChIM2bM0JEjRxQXF6fHHnus0nFGjx6tKVOmaPTo0Q32RG5Zlnbv3q2cnBzt2bPH0x4fH6++ffuqU6dOVf5+AgDqls06d2DBRaSkpCg5OVmvvPKKpDPd+O3atdPEiROrHMCamJio1NRUvfjii562t99+Ww8//LCKi4trNPjPbrcrOjpaRUVFioqKqrTu9OnT2rt3r6666qoGe9LzR/v27VOHDh20efNm3XDDDRfctr4/Q6fTqb///e+egcPSmct7Z48HAQBcugudv8/m9WWa9PR0jRs3TklJSUpOTlZWVpZKSko8YxfGjh2r+Ph4z+21aWlpmjt3ruea++7duzV9+nSlpaVxF0Ij5HQ6dezYMT3zzDPq06fPRYNIfSouLtbmzZv19ddf69SpU5LO9MJV/G42a9bMbIEA4Ke8DiOjRo1SQUGBZsyYoby8PPXs2VNr1671DGo9cOBApa7tZ555RjabTc8884wOHz6sVq1aKS0tzTNjJxqXL7/8UjfffLOuvfZaLVu2zHQ5kqS8vDzPeJCK26ebNWumlJQU9erVq9FfEgSAhs7ryzQmcJmmcbscn6FlWdq5c6dycnK0b98+T3tCQoL69Omj66+/nvEgAHCZXbbLNEBD5nA4PONBjh07JunMeJAuXbqoT58+io+PN1whAOBchBE0Cna7XZs2bdKWLVt0+vRpSWfGgyQmJio5OVnR0dGGKwQAVKfRhBEfuNqEalzKZ3fkyBHl5OTohx9+8IwHad68uWc8iC9MgAcA/s7nw0jFHTk1mUIdDVPFTKc1nU6/YlbYnJwc7d+/39N+5ZVXqk+fPrr22msZDwIAPsTnw0hQUJAiIiJUUFCg4OBgTkI+xLIslZaW6ujRo2rWrNlFb/V2OBz65ptv9NVXX+nEiROSzkweVzEepG3btvVRNgCgjvl8GLHZbIqLi9PevXsr/ZUM39GsWbNqZ/CVpKKiIs94kIovUAwLC/OMB7nQCG0AQMPn82FEOjN1eseOHT3fLAvfERwcXG2PyOHDhz3jQSrGlbRo0UJ9+vRRjx49GA8CAI1Eowgj0pnueuYZ8X1ut1vbt29XTk6ODh486Glv3769ZzyIzWYzWCEAoK41mjAC3/fTTz9p1apV+uWXXySdCZjdunVTnz59LngZBwDg2wgjaBByc3O1cuVKud1uhYeHKykpSb1791ZkZKTp0gAAlxlhBEZZlqUvvvhCn376qSSpW7duSktLq/FtvgAA30cYgTFut1tr1qzRli1bJEn9+/fXrbfeypgQAPAzhBEY4XQ69d5772nHjh2SpCFDhig5OdlwVQAAEwgjqHelpaV6++23dejQIQUFBemee+5Rp06dTJcFADCEMIJ6deLECb355ps6fvy4wsLCNHr0aLVr1850WQAAgwgjqDdHjhzR4sWLVVJSoujoaI0ZM0atWrUyXRYAwDDCCOrF7t279c4778jpdKpNmzZ64IEHuG0XACCJMIJ6kJubqxUrVsiyLF199dW67777FBoaarosAEADQRjBZXPuHCLdu3fXXXfdddFv5wUA+BfCCC4Lt9ut1atXa+vWrZKYQwQAUD3CCOqc0+nUsmXLtHPnTknMIQIAuDDCCOpUSUmJ3n77bR0+fJg5RAAANUIYQZ1hDhEAQG0QRlAnmEMEAFBbhBFcMuYQAQBcCsIILglziAAALhVhBLViWZY+//xzffbZZ5KYQwQAUHuEEXjt3DlEBgwYoFtuuYU5RAAAtUIYgVccDofee+89zxwiQ4cOVe/evQ1XBQDwZYQR1BhziAAALgfCCGrk+PHjeuutt3T8+HGFh4dr9OjRSkhIMF0WAKARIIzgos6dQ+TBBx9UTEyM6bIAAI0EYQQXtGvXLr377rvMIQIAuGwII6jWN998o5UrVzKHCADgsiKM4DzMIQIAqE+EEVTCHCIAgPpGGIHH2XOI2Gw2DRkyhDlEAACXHWEEks6fQ2TkyJG6/vrrTZcFAPADhBEwhwgAwCjCiJ87fPiwFi9erNLSUjVr1kxjxoxhDhEAQL0ijPixc+cQGTNmjJo2bWq6LACAnyGM+Kmz5xDp0KGD7r33XuYQAQAYQRjxM+fOIdKjRw+lpaUxhwgAwBjCiB9hDhEAQENEGPETDodDy5Yt065du5hDBADQoBBG/EBJSYkWL16sI0eOMIcIAKDBIYw0csePH9ebb76pEydOMIcIAKBBIow0YswhAgDwBYSRRmrnzp1atmwZc4gAABo8wkgjtHXrVq1atYo5RAAAPoEw0ohYlqV169Zp3bp1kphDBADgGwgjjYTb7daqVav0zTffSJIGDhyom2++mTlEAAANHmGkETh3DpGhQ4cqKSnJdFkAANQIYcTHHTx4UO+//76OHTvGHCIAAJ9EGPFR5eXl+vTTT7Vx40ZZlqWmTZvqvvvuYw4RAIDPIYz4oMOHD2v58uUqLCyUdGag6uDBgxUeHm64MgAAvEcY8SHl5eVat26dvvzyS09vyLBhw3TdddeZLg0AgFojjPiII0eOaPny5SooKJAkdevWTXfccYciIiIMVwYAwKUhjDRwLpdLn3/+ub744gtZlqWIiAgNGzZMnTp1Ml0aAAB1gjDSgOXl5Wn58uXKz8+XJHXp0kVDhgxRkyZNDFcGAEDdIYw0QC6XS1988YW++OILud1uRUREaOjQoerSpYvp0gAAqHOEkQYmPz9fy5cvV15eniSpU6dOuvPOO+kNAQA0WoSRBsLtdmv9+vVat26d3G63wsPDPb0hTOkOAGjMCCMNwNGjR/X+++/ryJEjkqTrrrtOw4YNU9OmTQ1XBgDA5UcYMcjtdmvDhg367LPP5HK5FBYWpiFDhqhbt270hgAA/AZhxJDCwkItX75chw8fliR17NhRaWlpioyMNFwZAAD1izBSz9xut3JycvTJJ5/I5XIpNDRUd9xxh3r06EFvCADALxFG6tGxY8e0fPlyHTp0SJJ0zTXXKC0tTVFRUYYrAwDAnIDa7DR//ny1b99eYWFhSklJ0aZNmy64/S+//KIJEyYoLi5OoaGhuvbaa7VmzZpaFeyLLMtSTk6OFixYoEOHDikkJERpaWl64IEHCCIAAL/ndc/I0qVLlZ6ergULFiglJUVZWVkaPHiwduzYodatW5+3vcPh0G233abWrVtr2bJlio+P1/79+9WsWbO6qL/BO378uN5//30dOHBAknT11VfrrrvuUnR0tOHKAABoGGyWZVne7JCSkqLevXtr3rx5ks6MgUhISNCkSZOUkZFx3vYLFizQ7NmztX37dgUHB9eqSLvdrujoaBUVFflMT4JlWdq0aZP+9re/qby8XCEhIbrtttuUmJjI2BAAgF+o6fnbq8s0DodDW7ZsUWpq6j8PEBCg1NRUbdy4scp9VqxYob59+2rChAmKjY1V165dNWvWLLlcrmpfp6ysTHa7vdLiS06cOKH/+Z//0dq1a1VeXq727dvr8ccfV1JSEkEEAIBzeHWZprCwUC6XS7GxsZXaY2NjtX379ir32bNnjz755BONGTNGa9as0e7du/Wv//qvcjqdmjlzZpX7ZGZm6tlnn/WmtAbBsix9/fXX+vjjj+V0OhUcHKzbbruNEAIAwAVc9rtp3G63WrdurYULFyowMFCJiYk6fPiwZs+eXW0YmTZtmtLT0z3P7Xa7EhISLnepl+SXX37RihUrtHfvXknSlVdeqbvuukstWrQwXBkAAA2bV2EkJiZGgYGBnq+0r5Cfn682bdpUuU9cXJyCg4MVGBjoaevUqZPy8vLkcDgUEhJy3j6hoaEKDQ31pjRjLMvS1q1b9dFHH8nhcCgoKEipqalKTk6mNwQAgBrwasxISEiIEhMTlZ2d7Wlzu93Kzs5W3759q9ynf//+2r17t9xut6dt586diouLqzKI+JKioiK99dZbWrVqlRwOhxISEvTYY48pJSWFIAIAQA15fZkmPT1d48aNU1JSkpKTk5WVlaWSkhKNHz9ekjR27FjFx8crMzNTkvT4449r3rx5mjx5siZNmqRdu3Zp1qxZeuKJJ+r2ndQjy7KUm5urDz/8UGVlZQoKCtItt9yilJQUBQTUauoWAAD8ltdhZNSoUSooKNCMGTOUl5ennj17au3atZ5BrQcOHKh0Qk5ISNCHH36oJ598Ut27d1d8fLwmT56sqVOn1t27qEd2u10rV67U7t27JUlXXHGFhg8frpiYGMOVAQDgm7yeZ8SEhjDPiGVZ+vbbb/XBBx+orKxMgYGBuvnmm9W3b196QwAAqEJNz998N00NFBcXa9WqVdq5c6ckqW3bthoxYoRatWpluDIAAHwfYeQCLMvSd999pw8++ECnT59WQECAbrrpJvXv35/eEAAA6ghhpBonT57U6tWrPZO5xcXFafjw4edN+AYAAC4NYeQclmXphx9+0Jo1a3Tq1CkFBARo0KBB6t+/f6W5UgAAQN0gjJylpKREa9as0bZt2ySdmeZ+xIgR1U7oBgAALh1h5B+2bdum1atXq7S0VAEBARo4cKAGDhxIbwgAAJeZ34eR0tJSrVmzRj/88IMkqXXr1hoxYoTi4uIMVwYAgH/w6zCyfft2rVq1SiUlJbLZbBowYIBuvPFGBQX59Y8FAIB65bdnXafTqQ8++EAlJSVq1aqVhg8frvj4eNNlAQDgd/w2jAQHBystLU379u3TTTfdRG8IAACG+PUZ+JprrtE111xjugwAAPwa04gCAACjCCMAAMAowggAADCKMAIAAIwijAAAAKMIIwAAwCjCCAAAMIowAgAAjCKMAAAAowgjAADAKMIIAAAwijACAACMIowAAACjCCMAAMAowggAADCKMAIAAIwijAAAAKMIIwAAwCjCCAAAMIowAgAAjCKMAAAAowgjAADAKMIIAAAwijACAACMIowAAACjCCMAAMAowggAADCKMAIAAIwijAAAAKMIIwAAwCjCCAAAMIowAgAAjCKMAAAAowgjAADAKMIIAAAwijACAACMIowAAACjCCMAAMAowggAADCKMAIAAIwijAAAAKMIIwAAwCjCCAAAMIowAgAAjCKMAAAAowgjAADAKMIIAAAwijACAACMIowAAACjCCMAAMAowggAADCKMAIAAIwijAAAAKMIIwAAwCjCCAAAMIowAgAAjCKMAAAAo2oVRubPn6/27dsrLCxMKSkp2rRpU432W7JkiWw2m0aMGFGblwUAAI2Q12Fk6dKlSk9P18yZM7V161b16NFDgwcP1tGjRy+43759+/Tb3/5WAwcOrHWxAACg8fE6jMydO1ePPvqoxo8fr86dO2vBggWKiIjQ66+/Xu0+LpdLY8aM0bPPPqurr776kgoGAACNi1dhxOFwaMuWLUpNTf3nAQIClJqaqo0bN1a733/+53+qdevWevjhh2v0OmVlZbLb7ZUWAADQOHkVRgoLC+VyuRQbG1upPTY2Vnl5eVXus379ev35z3/WokWLavw6mZmZio6O9iwJCQnelAkAAHzIZb2bpri4WA899JAWLVqkmJiYGu83bdo0FRUVeZaDBw9exioBAIBJQd5sHBMTo8DAQOXn51dqz8/PV5s2bc7b/qefftK+ffuUlpbmaXO73WdeOChIO3bsUIcOHc7bLzQ0VKGhod6UBgAAfJRXPSMhISFKTExUdna2p83tdis7O1t9+/Y9b/vrr79e3333nXJzcz3LXXfdpZtvvlm5ublcfgEAAN71jEhSenq6xo0bp6SkJCUnJysrK0slJSUaP368JGns2LGKj49XZmamwsLC1LVr10r7N2vWTJLOawcAAP7J6zAyatQoFRQUaMaMGcrLy1PPnj21du1az6DWAwcOKCCAiV0BAEDN2CzLskwXcTF2u13R0dEqKipSVFSU6XIAAEAN1PT8TRcGAAAwijACAACMIowAAACjCCMAAMAowggAADCKMAIAAIwijAAAAKMIIwAAwCjCCAAAMIowAgAAjCKMAAAAowgjAADAKMIIAAAwijACAACMIowAAACjCCMAAMAowggAADCKMAIAAIwijAAAAKMIIwAAwCjCCAAAMIowAgAAjCKMAAAAowgjAADAKMIIAAAwijACAACMIowAAACjCCMAAMAowggAADCKMAIAAIwijAAAAKMIIwAAwCjCCAAAMIowAgAAjCKMAAAAowgjAADAKMIIAAAwijACAACMIowAAACjCCMAAMAowggAADCKMAIAAIwijAAAAKMIIwAAwCjCCAAAMIowAgAAjCKMAAAAowgjAADAKMIIAAAwijACAACMIowAAACjCCMAAMAowggAADCKMAIAAIwijAAAAKMIIwAAwCjCCAAAMIowAgAAjCKMAAAAowgjAADAKMIIAAAwijACAACMIowAAACjCCMAAMAowggAADCKMAIAAIwijAAAAKMIIwAAwKhahZH58+erffv2CgsLU0pKijZt2lTttosWLdLAgQPVvHlzNW/eXKmpqRfcHgAA+Bevw8jSpUuVnp6umTNnauvWrerRo4cGDx6so0ePVrn9Z599ptGjR+vTTz/Vxo0blZCQoNtvv12HDx++5OIBAIDvs1mWZXmzQ0pKinr37q158+ZJktxutxISEjRp0iRlZGRcdH+Xy6XmzZtr3rx5Gjt2bI1e0263Kzo6WkVFRYqKivKmXAAAYEhNz99e9Yw4HA5t2bJFqamp/zxAQIBSU1O1cePGGh2jtLRUTqdTLVq0qHabsrIy2e32SgsAAGicvAojhYWFcrlcio2NrdQeGxurvLy8Gh1j6tSpatu2baVAc67MzExFR0d7loSEBG/KBAAAPqRe76Z54YUXtGTJEv31r39VWFhYtdtNmzZNRUVFnuXgwYP1WCUAAKhPQd5sHBMTo8DAQOXn51dqz8/PV5s2bS647+9//3u98MIL+tvf/qbu3btfcNvQ0FCFhoZ6UxoAAPBRXvWMhISEKDExUdnZ2Z42t9ut7Oxs9e3bt9r9XnrpJT333HNau3atkpKSal8tAABodLzqGZGk9PR0jRs3TklJSUpOTlZWVpZKSko0fvx4SdLYsWMVHx+vzMxMSdKLL76oGTNmaPHixWrfvr1nbEnTpk3VtGnTOnwrAADAF3kdRkaNGqWCggLNmDFDeXl56tmzp9auXesZ1HrgwAEFBPyzw+XVV1+Vw+HQr371q0rHmTlzpv7jP/7j0qoHAAA+z+t5RkxgnhEAAHzPZZlnBAAAoK4RRgAAgFGEEQAAYBRhBAAAGEUYAQAARhFGAACAUYQRAABgFGEEAAAYRRgBAABGEUYAAIBRhBEAAGAUYQQAABhFGAEAAEYRRgAAgFGEEQAAYBRhBAAAGEUYAQAARhFGAACAUYQRAABgFGEEAAAYRRgBAABGEUYAAIBRhBEAAGAUYQQAABhFGAEAAEYRRgAAgFGEEQAAYBRhBAAAGEUYAQAARhFGAACAUYQRAABgFGEEAAAYRRgBAABGEUYAAIBRhBEAAGAUYQQAABhFGAEAAEYRRgAAgFGEEQAAYBRhBAAAGEUYAQAARhFGAACAUYQRAABgFGEEAAAYRRgBAABGEUYAAIBRhBEAAGAUYQQAABhFGAEAAEYRRgAAgFGEEQAAYBRhBAAAGEUYAQAARhFGAACAUYQRAABgFGEEAAAYRRgBAABGEUYAAIBRhBEAAGAUYQQAABhFGAEAAEYRRgAAgFGEEQAAYBRhBAAAGBVkugCjjh2T3G4pKKjyEhgoBZDTAACoD/4dRu66S9qwoep1AQHnh5TLuQQHX9r+gYGSzVazRar5tib2DwiovJzbdrFtKl4DAOAT/DuMWFb169xuyeE4s8D31EWoudR9zg5FZ4ewc9sudf3lOGZ166varr63qctj1+S16nI9r1E36xvSupqsr4mGcIwnn5Tat7/0OmrBv8PIhg1nAonLJZWX135xOi9t/7o6pmVVv0gXXt/Qtne7/7nURsXn6nJd2u8IAPiL0aN9K4zMnz9fs2fPVl5ennr06KFXXnlFycnJ1W7/7rvvavr06dq3b586duyoF198UUOHDq110XXKZvvnpQ40TFUFFLe74bedXf+5j6tqq8njy7VtTfarrq0+t6nLY9fktepyfX28xoXW+8q6hlZPXXzul6o+XkOS4uPr53Wq4PUZeOnSpUpPT9eCBQuUkpKirKwsDR48WDt27FDr1q3P237Dhg0aPXq0MjMzNWzYMC1evFgjRozQ1q1b1bVr1zp5E2jkzh5LAgBodGyW5V3kSklJUe/evTVv3jxJktvtVkJCgiZNmqSMjIzzth81apRKSkq0atUqT1ufPn3Us2dPLViwoEavabfbFR0draKiIkVFRXlTLgAAMKSm52+v/tR0OBzasmWLUlNT/3mAgAClpqZq48aNVe6zcePGSttL0uDBg6vdXpLKyspkt9srLQAAoHHyKowUFhbK5XIpNja2UntsbKzy8vKq3CcvL8+r7SUpMzNT0dHRniUhIcGbMgEAgA9pkBfhp02bpqKiIs9y8OBB0yUBAIDLxKsBrDExMQoMDFR+fn6l9vz8fLVp06bKfdq0aePV9pIUGhqq0NBQb0oDAAA+yquekZCQECUmJio7O9vT5na7lZ2drb59+1a5T9++fSttL0kff/xxtdsDAAD/4vWtvenp6Ro3bpySkpKUnJysrKwslZSUaPz48ZKksWPHKj4+XpmZmZKkyZMna9CgQZozZ47uvPNOLVmyRF9//bUWLlxYt+8EAAD4JK/DyKhRo1RQUKAZM2YoLy9PPXv21Nq1az2DVA8cOKCAs+aD6NevnxYvXqxnnnlGTz31lDp27Kjly5czxwgAAJBUi3lGTGCeEQAAfM9lmWcEAACgrhFGAACAUYQRAABgFGEEAAAY5fXdNCZUjLHlO2oAAPAdFefti90r4xNhpLi4WJL4jhoAAHxQcXGxoqOjq13vE7f2ut1uHTlyRJGRkbLZbHV2XLvdroSEBB08eJBbhhsAPo+Gh8+kYeHzaFj4PC7OsiwVFxerbdu2leYgO5dP9IwEBAToiiuuuGzHj4qK4hepAeHzaHj4TBoWPo+Ghc/jwi7UI1KBAawAAMAowggAADDKr8NIaGioZs6cqdDQUNOlQHweDRGfScPC59Gw8HnUHZ8YwAoAABovv+4ZAQAA5hFGAACAUYQRAABgFGEEAAAY5ddhZP78+Wrfvr3CwsKUkpKiTZs2mS7JL2VmZqp3796KjIxU69atNWLECO3YscN0WfiHF154QTabTVOmTDFdit86fPiwHnzwQbVs2VLh4eHq1q2bvv76a9Nl+S2Xy6Xp06frqquuUnh4uDp06KDnnnvuot+/gur5bRhZunSp0tPTNXPmTG3dulU9evTQ4MGDdfToUdOl+Z1169ZpwoQJysnJ0ccffyyn06nbb79dJSUlpkvze5s3b9Zrr72m7t27my7Fb504cUL9+/dXcHCwPvjgA23btk1z5sxR8+bNTZfmt1588UW9+uqrmjdvnn788Ue9+OKLeumll/TKK6+YLs1n+e2tvSkpKerdu7fmzZsn6cz33yQkJGjSpEnKyMgwXJ1/KygoUOvWrbVu3TrdeOONpsvxWydPntQNN9ygP/3pT/qv//ov9ezZU1lZWabL8jsZGRn68ssv9cUXX5guBf8wbNgwxcbG6s9//rOnbeTIkQoPD9ebb75psDLf5Zc9Iw6HQ1u2bFFqaqqnLSAgQKmpqdq4caPByiBJRUVFkqQWLVoYrsS/TZgwQXfeeWel/09Q/1asWKGkpCTde++9at26tXr16qVFixaZLsuv9evXT9nZ2dq5c6ck6e9//7vWr1+vIUOGGK7Md/nEF+XVtcLCQrlcLsXGxlZqj42N1fbt2w1VBelMD9WUKVPUv39/de3a1XQ5fmvJkiXaunWrNm/ebLoUv7dnzx69+uqrSk9P11NPPaXNmzfriSeeUEhIiMaNG2e6PL+UkZEhu92u66+/XoGBgXK5XHr++ec1ZswY06X5LL8MI2i4JkyYoO+//17r1683XYrfOnjwoCZPnqyPP/5YYWFhpsvxe263W0lJSZo1a5YkqVevXvr++++1YMECwogh77zzjt566y0tXrxYXbp0UW5urqZMmaK2bdvymdSSX4aRmJgYBQYGKj8/v1J7fn6+2rRpY6gqTJw4UatWrdLnn3+uK664wnQ5fmvLli06evSobrjhBk+by+XS559/rnnz5qmsrEyBgYEGK/QvcXFx6ty5c6W2Tp066b333jNUEX73u98pIyND999/vySpW7du2r9/vzIzMwkjteSXY0ZCQkKUmJio7OxsT5vb7VZ2drb69u1rsDL/ZFmWJk6cqL/+9a/65JNPdNVVV5kuya/deuut+u6775Sbm+tZkpKSNGbMGOXm5hJE6ln//v3Pu9V9586duvLKKw1VhNLSUgUEVD59BgYGyu12G6rI9/llz4gkpaena9y4cUpKSlJycrKysrJUUlKi8ePHmy7N70yYMEGLFy/W+++/r8jISOXl5UmSoqOjFR4ebrg6/xMZGXneeJ0mTZqoZcuWjOMx4Mknn1S/fv00a9Ys3Xfffdq0aZMWLlyohQsXmi7Nb6Wlpen5559Xu3bt1KVLF33zzTeaO3eu/uVf/sV0ab7L8mOvvPKK1a5dOyskJMRKTk62cnJyTJfklyRVubzxxhumS8M/DBo0yJo8ebLpMvzWypUrra5du1qhoaHW9ddfby1cuNB0SX7NbrdbkydPttq1a2eFhYVZV199tfX0009bZWVlpkvzWX47zwgAAGgY/HLMCAAAaDgIIwAAwCjCCAAAMIowAgAAjCKMAAAAowgjAADAKMIIAAAwijACAACMIowAAACjCCMAAMAowggAADCKMAIAAIz6fyZeMbrIxZRtAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "from torchvision import transforms\n",
    "from Module.Utils import *\n",
    "\n",
    "trans = transforms.Compose(\n",
    "    [\n",
    "        transforms.ToTensor(),\n",
    "        transforms.Resize(size=(224,224))\n",
    "    ]\n",
    ")\n",
    "\n",
    "device = 'cuda' if torch.cuda.is_available() else 'cpu'\n",
    "    \n",
    "net = VGGNet_11().to(device)\n",
    "net.apply(module_init)\n",
    "\n",
    "epochs = 10\n",
    "lr = 0.05\n",
    "batch_size = 128\n",
    "\n",
    "data_num, train_iter, test_iter = load_data_FMNIST('../data', batch_size, trans)\n",
    "loss = nn.CrossEntropyLoss()\n",
    "optimizer = torch.optim.SGD(net.parameters(), lr)\n",
    "\n",
    "train(net, train_iter, test_iter, epochs, optimizer, loss, True)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "torch",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.14"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
